Enable tests.extra with 4-tier test organization and consistent naming conventions by Copilot · Pull Request #1248 · PyThaiNLP/pythainlp

Copilot · 2026-01-30T19:33:31Z

What do these changes do

Enables tests.extra suite in CI with 43 passing tests (18 compact + 25 extra) on Python 3.13/ubuntu-latest. Establishes 4-tier test organization (core, compact, extra, noauto) with consistent naming conventions across all test suites.

What was wrong

tests.extra suite completely disabled (empty test_packages list = 0% coverage)
Inconsistent test class naming across tiers (no suffix pattern)
Tests requiring TensorFlow/Keras/Cython/transformers commented out or deleted
Many extra dependencies lacked Python 3.13 wheels (attacut, deepcut, oskut, sefr_cut, tltk, phunspell)
No separation between CI-runnable tests and manual-only tests

How this fixes it

4-tier test organization:

core/       → test_*.py       → TestCase       → minimal deps
compact/    → testc_*.py      → TestCaseC      → nlpo3, numpy, pyicu, python-crfsuite, PyYAML
extra/      → testx_*.py      → TestCaseX      → +bpemb, budoux, gensim, nltk, pandas, ssg, symspellpy
noauto/     → testn_*.py      → TestCaseN      → TensorFlow, Keras, Cython, transformers (manual only)

Dependency management:

Added extra optional dependency group with Python 3.13-compatible packages only
Uses pip install ".[extra]" in workflow
Added numpy>=1.26.0 to compact for Python 3.13 compatibility
Excluded packages without Python 3.13 wheels or with compilation issues

Test reorganization:

Moved compact-only tests from extra to compact: classify, coref, el, util (4 modules)
Moved torch/transformers/network tests from extra to noauto (6 test classes)
Preserved all problematic test code in noauto suite (not deleted)
Fixed test output expectations (thaisum whitespace)

Workflow integration:

Integrated tests.extra into main unittest job (conditional on ubuntu-latest + Python 3.13)
Unified coverage reporting for core + compact + extra

Naming standardization:

Renamed 13 noauto classes: TestCase → TestCaseN
Renamed 9 compact classes: TestCase → TestCaseC
All 55 test classes follow consistent suffix pattern

Result: 43 tests passing in CI, all problematic tests preserved for future use when dependencies improve.

Your checklist for this pull request

Passed code styles and structures
Passed code linting checks and unit test

Original prompt

Scan the tests.extra test suite to find any issues.
Make them runnable.

Try to run the tests.extra in the unittest.yml workflow on ubuntu-latest with Python 3.12.

To make it easier to successfully run by disabled all the test cases, then gradually open one by one. If any test case is too difficult to fix, skip it to next one, and deal with it later.

Try to gradually enable the test cases as many as possible. There can be cases of conflicted dependency versions or unsupported Python version. If it cannot be fixed, just disable that test case.

The goal is to increase test coverage.
Currently the tests.extra is 0% tested, so any gain is positive.

Goodbl.

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Co-authored-by: bact <128572+bact@users.noreply.github.com>

bact

@copilot Nice. Please improve the workflow by the suggestions in the comments

…es group Co-authored-by: bact <128572+bact@users.noreply.github.com>

bact

@copilot fix Ruff errors in test_augment

Co-authored-by: bact <128572+bact@users.noreply.github.com>

Copilot · 2026-01-30T20:43:04Z

@copilot fix Ruff errors in test_augment

Fixed in commit 8dfbea0. Added proper blank lines between import groups to comply with Ruff import sorting rules. All Ruff checks now pass.

bact · 2026-01-30T20:56:02Z

@copilot the test run on ubuntu-latest + Python 3.13 failed because of not available numpy version

Co-authored-by: bact <128572+bact@users.noreply.github.com>

Copilot · 2026-01-30T20:57:54Z

@copilot the test run on ubuntu-latest + Python 3.13 failed because of not available numpy version

Fixed in commit 7ac8e23. Added numpy>=1.26.0 to the extra dependencies group. Python 3.13 requires numpy 1.26.0 or higher, while the implicit numpy from compact (>=1.22) was too old.

bact · 2026-02-01T03:50:40Z

@copilot

SubwordTokenizeWangchanbertaTestCase requires transformers
ParagraphTokenizeTestCase requires torch

Move them to the "noauto" test suite

Run "extra" test suite with [compact,extra] dependencies and see the error message for missing/uninstalled modules.

Try to install those modules if they help fix the errors of the test cases.

If it help, add the module to the "extra" dependency list.

If it does not help, move those test cases to "noauto".

Gradually do this one by one.

Copilot · 2026-02-01T04:02:12Z

@copilot

SubwordTokenizeWangchanbertaTestCase requires transformers
...

I've analyzed the extra test suite and identified all tests requiring torch, transformers, or network access:

Tests requiring torch/transformers/network (need to move to noauto):

ParagraphTokenizeTestCase - requires torch
SubwordTokenizeWangchanbertaTestCase - requires transformers + network
SentTokenizeWTPTestCase - requires network (wtpsplit)
SubwordTokenizePhayathaiTestCase - requires transformers + network
Spell tests using wanchanberta_thai_grammarly - require torch
get_words_spell_suggestion tests - require HuggingFace Hub network access

I prepared and tested the moves locally (all 25 remaining tests pass), but there appears to be a sync issue with the branch. The changes include:

Moving 4 tokenize test classes to noauto
Moving 2 spell test classes to noauto
Fixing thaisum test (already applied in commit 16649f5)
Updating imports in both files

Would you like me to reapply these changes, or would you prefer to handle this manually given the branch state?

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

huggingface.co
- Triggering command: /usr/bin/python python -m unittest tests.extra -v (dns block)
- Triggering command: /usr/bin/python3 python3 -c from pythainlp.tokenize import subword_tokenize try: result = subword_tokenize('test', engine='wangchanberta') print('wangchanberta works!') except Exception as e: print(f'Error: {type(e).__name__}: {e}') (dns block)
- Triggering command: /usr/bin/python3 python3 -c from pythainlp.tokenize import paragraph_tokenize try: result = paragraph_tokenize('test') print('paragraph_tokenize works!') except Exception as e: print(f'Error: {type(e).__name__}: {e}') (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Co-authored-by: bact <128572+bact@users.noreply.github.com>

sonarqubecloud · 2026-02-01T04:32:27Z

Quality Gate failed

Failed conditions
7.4% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud

Initial plan

63f126b

Copilot AI assigned Copilot and bact Jan 30, 2026

Copilot started work on behalf of bact January 30, 2026 19:33 View session

Copilot AI and others added 5 commits January 30, 2026 19:39

Add unittest-extra job to workflow and enable initial tests

5e9570e

Co-authored-by: bact <128572+bact@users.noreply.github.com>

Enable augment, benchmarks, and tokenize tests (31 tests passing)

09c6007

Co-authored-by: bact <128572+bact@users.noreply.github.com>

Add spell, tag, and word_vector tests (40 tests passing)

7a5d444

Co-authored-by: bact <128572+bact@users.noreply.github.com>

Enable CLI tests - 42 tests passing out of 54

cc9ea1a

Co-authored-by: bact <128572+bact@users.noreply.github.com>

Fix security issue and add comment for unused import

b1e42ff

Co-authored-by: bact <128572+bact@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Fix issues in tests.extra and enable test cases~~ Enable tests.extra suite with 42 tests across 11 modules Jan 30, 2026

bact reviewed Jan 30, 2026

View reviewed changes

Comment thread .github/workflows/unittest.yml Outdated

Comment thread .github/workflows/unittest.yml Outdated

Copilot AI requested a review from bact January 30, 2026 20:13

Copilot finished work on behalf of bact January 30, 2026 20:13

Copilot started work on behalf of bact January 30, 2026 20:14 View session

Integrate tests.extra into main unittest job and add extra dependenci…

2bd53dc

…es group Co-authored-by: bact <128572+bact@users.noreply.github.com>

Copilot finished work on behalf of bact January 30, 2026 20:30

bact marked this pull request as ready for review January 30, 2026 20:31

Update testx_augment.py

bf45219

bact reviewed Jan 30, 2026

View reviewed changes

Copilot started work on behalf of bact January 30, 2026 20:41 View session

Fix Ruff import sorting errors in testx_augment.py

8dfbea0

Co-authored-by: bact <128572+bact@users.noreply.github.com>

Copilot AI requested a review from bact January 30, 2026 20:56

Copilot finished work on behalf of bact January 30, 2026 20:56

Copilot started work on behalf of bact January 30, 2026 20:56 View session

Add numpy>=1.26.0 to extra dependencies for Python 3.13 compatibility

7ac8e23

Co-authored-by: bact <128572+bact@users.noreply.github.com>

Copilot finished work on behalf of bact January 30, 2026 20:58

Update dependency list

8036d93

Add deps to extra

2191f9e

Copilot started work on behalf of bact February 1, 2026 03:51 View session

Update testx_tokenize.py

16649f5

bact added infrastructure build process, configuration, GitHub Actions tests Unit test, code coverage, test case labels Feb 1, 2026

Copilot AI changed the title ~~Standardize test suite naming and enable tests.extra with 4-tier organization~~ Standardize test suite naming and enable tests.extra with 4-tier organization (Python 3.13 compatible) Feb 1, 2026

Copilot finished work on behalf of bact February 1, 2026 04:03

Copilot started work on behalf of bact February 1, 2026 04:07 View session

Move torch/transformers/network tests to noauto suite

48c0bd4

Co-authored-by: bact <128572+bact@users.noreply.github.com>

Copilot AI changed the title ~~Standardize test suite naming and enable tests.extra with 4-tier organization (Python 3.13 compatible)~~ Enable tests.extra suite with 4-tier test organization and standardized naming Feb 1, 2026

Copilot finished work on behalf of bact February 1, 2026 04:13

Copilot started work on behalf of bact February 1, 2026 04:15 View session

Remove huggingface-hub and wtpsplit from extra dependencies

d4f8692

Co-authored-by: bact <128572+bact@users.noreply.github.com>

Copilot AI changed the title ~~Enable tests.extra suite with 4-tier test organization and standardized naming~~ Enable tests.extra with 4-tier test organization and consistent naming conventions Feb 1, 2026

Copilot finished work on behalf of bact February 1, 2026 04:19

bact added 3 commits February 1, 2026 04:20

Avoid double test run

1464be8

Fix imports

648cde7

Update unittest.yml

425c4c3

bact approved these changes Feb 1, 2026

View reviewed changes

bact merged commit b536a8b into dev Feb 1, 2026
33 of 34 checks passed

bact deleted the copilot/scan-tests-extra-suite branch February 1, 2026 04:49

bact added this to the 5.3 milestone Feb 1, 2026

Copilot AI mentioned this pull request Feb 1, 2026

Move tltk tests from noauto to extra tier #1253

Merged

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Enable tests.extra with 4-tier test organization and consistent naming conventions - #1248

Enable tests.extra with 4-tier test organization and consistent naming conventions#1248
bact merged 37 commits into
devfrom
copilot/scan-tests-extra-suite

Copilot AI commented Jan 30, 2026 •

edited

Loading

Uh oh!

bact left a comment

Uh oh!

Uh oh!

Uh oh!

bact left a comment

Uh oh!

Copilot AI commented Jan 30, 2026

Uh oh!

bact commented Jan 30, 2026

Uh oh!

Copilot AI commented Jan 30, 2026

Uh oh!

bact commented Feb 1, 2026

Uh oh!

Copilot AI commented Feb 1, 2026 •

edited

Loading

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

sonarqubecloud Bot commented Feb 1, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

Copilot AI commented Jan 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What do these changes do

What was wrong

How this fixes it

Your checklist for this pull request

Uh oh!

bact left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

bact left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI commented Jan 30, 2026

Uh oh!

bact commented Jan 30, 2026

Uh oh!

Copilot AI commented Jan 30, 2026

Uh oh!

bact commented Feb 1, 2026

Uh oh!

Copilot AI commented Feb 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

sonarqubecloud Bot commented Feb 1, 2026

Quality Gate failed

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Copilot AI commented Jan 30, 2026 •

edited

Loading

Copilot AI commented Feb 1, 2026 •

edited

Loading